Can AI deliver appropriate support for diverse student profiles?: A large-scale evaluation

Abstract

Large language models (LLMs) have the potential to offer a novel approach to student support through learning analytics (LA)–informed advising. This study examines such potential by exploring whether current LLMs can adaptively recommend appropriate support across student needs, characteristics, and contexts. To do that, we generated 4,500 synthetic student vignettes using three LLMs (GPT-5-mini, Mistral-Medium-2508, Qwen-Plus). Each vignette included a single behavioral student trait, classified as either high- or low-performing, and one positive or negative LA indicator describing student learning behavior. These were randomly selected from pre-defined literature-informed lists to provide ..